fix: configure Kubernetes DNS search domains in mesh - #539
Conversation
Signed-off-by: Sam <55770131+Sam6734@users.noreply.github.com>
✅ Deploy Preview for interlink-dev ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
There was a problem hiding this comment.
Pull request overview
Note
Copilot couldn't run its full agentic review because no GitHub Actions runner was available. Make sure your repository has a runner available to run Copilot's review, or add a copilot-setup-steps.yml file specifying one with the runs-on attribute. See the docs for more details.
Fixes DNS resolution for Dask workers in the mesh by ensuring Kubernetes CoreDNS is preferred over host/public resolvers and by removing an incorrect LOCALDOMAIN override that interfered with name resolution.
Changes:
- Reorders
/etc/resolv.confto place Kubernetes DNS ({{.DNSServiceIP}}) before the host DNS entry. - Removes the
LOCALDOMAINexport and the associated temporaryresolv.confgeneration that incorrectly set a filename as a search domain.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| mkdir -p /tmp/etc-override | ||
| echo "search default.svc.cluster.local svc.cluster.local cluster.local" > /tmp/etc-override/resolv.conf | ||
| echo "nameserver $HOST_DNS" >> /tmp/etc-override/resolv.conf | ||
| echo "nameserver {{.DNSServiceIP}}" >> /tmp/etc-override/resolv.conf | ||
| echo "nameserver $HOST_DNS" >> /tmp/etc-override/resolv.conf | ||
| echo "nameserver 1.1.1.1" >> /tmp/etc-override/resolv.conf | ||
| echo "nameserver 8.8.8.8" >> /tmp/etc-override/resolv.conf |
Signed-off-by: Sam <55770131+Sam6734@users.noreply.github.com>
Signed-off-by: Diego Ciangottini <dciangot@cern.ch>
|
Hi @Sam6734 , thank you for contributing! I'm resolving a conflict with a previous PR, meanwhile, can I ask you what is the use case for having a full mesh connectivity with dask? We are interested to know, since most of the cases we are aware of, are fine with just portforwarding the head node ports. |
The Dask workers could start their WireGuard tunnels, but they could not resolve the Kubernetes name of the Dask scheduler. The mesh script treated a DNS configuration filename as a search domain. They eventually failed with:
OSError: Timed out trying to connect to tls://dask-daskclusterid.cmsaf-dev:8786 after 30 sThis change removes the LOCALDOMAIN override and places k8s CoreDNS before the host and public DNS servers in /etc/resolv.conf. Workers can now resolve and connect to the scheduler, while retaining their existing DNS servers as fallbacks.